IntellIgent VIsual InspectIon of BottlIng productIon lIne through neural network

This paper presents research done on developing an intelligent visual inspection system for automatic inspection of bottling production line. The objective of this research is to enhance on modeling, integrating, and implementation of intelligent visual inspection system in the process of quality control in industrial manufacturing. The system will inspect each individual product in real-time process. Levenberg-Marquardt backpropagation neural network has been applied for this system to differentiate between acceptable and unacceptable product, for example, the misplacement of a bottle cap. A simulation of the operation was attempted in the Robotics System Laboratory of Industrial Computing Department, Universiti Kebangsaan Malaysia. The experiments were done by using developed software (Real-TIVI) and hardware, i.e. conveyor belt, adjustable halogen lamp, personal computer, web camera (webcam) to capture the image, and plastic bottle as an object of visual inspection. From this experiment, the maximum regular speed of a rotating object was 106 rpm. The result shows the system is accurate to determine between acceptable (normal) and non-acceptable (no cap or misplace of cap) during the maximum speed when the distance between webcam and the object was at 15 cm. keywords: Bottling Production Line, Neural Network, Quality Control, Visual Inspection 1.0 IntroductIon In industrial manufacturing, product inspection is an important step in the production process. Since product reliability is of utmost importance in most mass production facilities, 100 percent product inspection of all parts, subassemblies, and finished product is often being attempted. The most difficult task for inspection is inspecting by visual appearance. Visual inspection seeks to identify both functional and cosmetic defects. The visual inspection in most manufacturing process depends mainly on human operators whose performance is generally inadequate and variable. Advances in technology have resulted in better, cheaper image analysis equipment, which enable the use of affordable automated visual inspection system [1]. The major advantages of automatic operation are speed and diagnostic capabilities. There has been extensive research in the area of visual inspection system. These activities include, among others, delicate electronics component manufacturing, quality textile production, metal product finishing, glass manufacturing, machine parts, printing products and granite quality inspection, integrated circuits (IC) manufacturing and many others [29]. Visual inspection technology improves productivity and quality management and provides a competitive advantage to industries that employ this technology. The requirements for the design and development of a successful visual inspection system vary depending on the application domain and are related to the tasks to be accomplished, environment, speed, etc. For example, in visual inspection applications, the system must be able to differentiate between acceptable and unacceptable variations or defects in products, while in other applications, the system must enable users to solve guidance and alignment tasks or, verify measurement and assembly tasks. 1.1 proBlem statement There exists no industrial vision system capable of handling all tasks in every application field. Only once the requirements of a particular application domain are specified, then appropriate decisions for the design and development of the application can be taken [10]. The first problem to solve in automated visual inspection task is to understand what kind of information the visual inspection system is to retrieve and how this translates into measurements or features at the images. The artificial intelligence techniques have been applied for visual inspection to differentiate between acceptable and unacceptable (reject) products. This paper presents a neural network application to recognise between accepted and rejected products in the real-time visual inspection system, especially in bottling production line. Journal The Institution of Engineers, Malaysia (Vol. 68, No.4, December 2007) 57 (Date received: 19.2.2008) rIza sulaIman and anton s. praBuwono Journal The Institution of Engineers, Malaysia (Vol. 69, No.4, December 2008) 58 2. related work 2.1 Image processIng tools Image processing is usually performed within rectangles, circles or along lines and arcs. Image processing operators include filtering (e.g. smoothing, sharpening), edge detection, threshold, morphological operations, etc. Such operations can be used to improve image quality (e.g. reduce noise, improve contrast) and to enhance or separate certain image features (e.g. regions, edges) from the background. Image processing operations transform an input image to another image having the desired characteristics. Image analysis is related to the extraction and measurement of image features and transforms these image features to numbers, vectors, character strings, etc. For example, lines, regions, characters, holes, rips, tears can be gauged or counted. The ultimate goal of image analysis is geared towards pattern recognition, i.e. the extraction of features that can be used by classifiers to recognise or classify objects [10]. A comparative chart of some of the most popular image processing tools and a proposed technology (Real-TIVI) is given in the Table 1. software package library Visual programming command line dedicated hardware source code IPL Lib Yes Yes No Yes No Sherlock32/MVTools Yes Yes No Yes Yes Image-Pro plus Yes Yes No No No OPTIMAS Yes Yes No No No WiT Yes Yes Optional Yes No PC Image Flow Yes Yes Datacube Yes No Intel Image Processing Lib. Yes No MMX No HALCON Yes Yes No No No VISION97 Yes Yes (frame grabber) No No AdOculos Yes Yes No No MIL Yes Yes Matrox Yes No Rhapsody Yes No No Yes No Real-TIVI Yes Yes No No Yes table 1 : comparison of some of the most image processing tools and real-tIVI 2.2 neural network approach Neural networks are being successfully applied across a wide range of application domains in business, medicine, geology and physics to solve problems of prediction, classification and control. Neural networks are composed of a number of similar elementary processing units (neurons) connected together into a network [11]. Neurons are arranged in layers with the input data initialising the processing at the input layer. The processed data of each layer passes through the network towards the output layer. Neural networks adapt the weights of their neurons during a training period based on examples, often with a known desired solution (supervised training). After sufficient training, the neural network is able to relate the problem data to the appropriate solution spaces, i.e. generate input/ output relations, thus offering a viable solution to a new problem through examples [12]. Neural networks are capable of handling a variety of image classification tasks in industrial vision environments, ranging from simple gauging to advanced classification problems, such as fault detection, optical character recognition, operation prediction, engine monitoring and control, etc. They can be used either as standalone techniques e.g. wood [13], seam [3], and surface roughness inspection [14], or in conjunction with other methods (e.g. solder joint inspection) [15]. Neural networks have been applied in all classes of quality inspection, namely dimensional quality [16,15,17], surface quality [3,14,18,19], structural quality [20] and operational quality [21]. They are applicable in almost every situation where a relationship between input and output parameters exists, even in cases where this relationship is very complex and cannot be expressed or handled by mathematical or other modeling means. Table 2 summarises features of the most commonly used neural network tools and the proposed system. IntellIgent VIsual InspectIon of BottlIng productIon lIne through neural network Journal The Institution of Engineers, Malaysia (Vol. 69, No.4, December 2008) 59 3.0 sYstem desIgn and modelIng A software program for work for image processing in defect detection of real-time visual inspection system has been developed. The developed program is called Real-Time Intelligent Visual Inspection (Real-TIVI). The program will start at image acquisition and will go through a series of processes before the results can be output. Figure 1 shows the Real-TIVI software framework, which was developed. The process starts with image acquisition, where image will be captured, followed by preprocessing the captured image captured to reduce noises. Images are then enhanced to ease the analysing process. After the image have been enhanced, the edges of image are then determined. Lastly, according to the parameter of edges, the status of a bottle in the bottling process can be determined by using neural network and action can be taken to follow up this result [23]. 3.1 confIguratIon suBsYstem The content of a configuration subsystem is video configuration module. Video configuration module used to configure the information e.g. to choose proper webcam device, image brightness, image colour, etc. This module is also used to configure the information such as image size and image resolution. In this system, image size is fixed i.e. 352 x 280 pixels, and the image input format is in grayscale. 3.2 Image processIng suBsYstem Image processing subsystem consists of some modules as shown in Figure 1, i.e.: Figure 1 : real-tIVI software framework package name types of neural network Industrial applications package or library user Interface code generation/ dll Brain Maker Backpropagation Software package Graphical C Neuro Solution Recurrent backpropagation, Backpropagation through time Summaries of applications included Software package Graphical C++/DLL G2 NeurOnline Backpropagation, RBF, Rho, Auto associative Detailed petrochemical application included Software package Graphical object oriented SPRLIB ANNLIB Backpropagation, pseudoNewton, LevenbergMarquardt, Conjugate gradient descent, BFGS, Kohonen maps C/C++ Libraries ILIB C/C++ Libraries Neural Connection Multilayer perceptron, RBF, Kohonen, Bayesian Software package Graphical DataEngine, v.i,ADL Multilayer perceptron, Kohonen, Feature map, Fuzzy, Kohonen network Software package Graphical C++/DLL Trajan 3.0 Offers all of the above architectures and training algorithms Software package Graphical DLL Real-TIVI Backpropagation, LevenbergMarquardt Bottling machine, Modular Automation Production System (MAPS) Libraries Graphical Visual C++ table 2: summarise the features of the most commonly used neural network tools rIza sulaIman and anton s. praBuwono Journal The Institution of Engineers, Malaysia (Vol. 69, No.4, December 2008) 60 3.3 Image acquIsItIon module In the development of a vision system, image acquisition is the first and the most important step. Any deficiency of the initial image can cause a major problem while processing and analysing the image. Hardware equipment plays a very important role in acquiring the image with sufficient contrast and sharp focusing. Microsoft Visual C++ has been chosen as the development tools. Using the tools mentioned, there are several ways to acquire a video stream from a webcam. The most common ways are DirectX, QuickCAM or video for windows (VFW). A real-time series of image can be acquired using the tools mentioned. Each individual image was stored and further analysing can be carried on there after. 3.4 preprocessIng module After the image has been captured from the first stage, each image will go through the preprocessing stage to eliminate noise inside the image, to enhance the result of the output. Imaging sensor including camera-like devices, rarely have evenly illuminated image. Even in the absence of vignette (which causes off-axis rays to be lost by collision with the lens mount at large apertures – typically f/2 or greater) image brightness falls off rapidly away from the axis of the imaging lens [24]. 3.5 Image enhancement module After the noises have been removed at the preprocessing stage, the image is then processed to maximise the contrast to give optimum output for processing the edges [25]. In our case, since the colours of our target unit are in bright colour, whereas the background in dark colour, therefore to maximise the contrast of the image, a simple algorithm as below are being perform: IF (P old >threshold_value) THEN P new = P old + enhance_brightness ELSE P new = P old enhance_darkness IF (P new > 255) THEN P new = 255 IF (P new < 0) THEN P new = 0 The first part of the algorithm is used to maximise the contrast of two regions and the edge in the image. If the pixel value in a pixel is more than a threshold value, certain value of brightness (enhance_brightness) is added to the pixel. Whereas, when it is less than the threshold value, certain value of darkness value (enhance_darkness) is subtract from the pixel. Since the value of a pixel range from 0 – 255, therefore, the second part of the algorithm is used to ensure the pixel value after calculation are within the range. Figure 2 : architecture of back-propagation neural network IntellIgent VIsual InspectIon of BottlIng productIon lIne through neural network Journal The Institution of Engineers, Malaysia (Vol. 69, No.4, December 2008) 61 3.6 edge detectIon module Edge detection is the most important and basic part of image processing. The major edge detection methods are review from the signal processing and artificial intelligence point of view. Human vision system and computer vision shows that an object can be expressed by it edges, and many kinds of identification algorithm can be generalised easily from the edge expression. Edge can be define as the line of intersection between two surfaces, or in the software point of view, the notation of edge implies to the variation of brightness in a region of pixels. In this software, the edge detection methods using Vertical Sobel Operator with special modification for best suiting the application [26]. 3.7 edge analYsIng module After the edges have been detected from the edge detection stage, the parameter return from the edge detection result is being used for analysing and determining size of the object. The function is to define and measure height and width of the object under inspection. 3.8 neural network module Levenberg-Marquardt back-propagation neural network has been chosen as support tool to determine between acceptable and unacceptable plastic bottle. The system is trained to get the best performance between target and output. After training, mathematical equation from back-propagation neural network model integrates the equation into Microsoft Visual C++ platform. This will recognise inspected bottle status in a real-time process. The analysis results are divided into options ‘accept’ or ‘reject’ product. Figure 2 shows the architecture of back-propagation neural network as a model in the system. The steps and mathematical formulae below were used to get a model of back-propagation neural network in Matlab. Each input unit (x i , i = 1,2,3,...n) receives a signal xi and send the signals to all of units in the next layer (hidden layer). Each unit of hidden layer (z_in j , j = 1,2,3,...p) will be added the weighted input signals: